This study centered around two research questions:
1. How does the location of an educational electronic device change interactions with it?
2. How does an educational electronic device's level of interest in a participant change interactions?
The experiment had a 2x2 matrix study design where students interacted with an arduino robot who was either interested or uninterested, and either on the working area (embedded) or separated from the working area (external). The interested device would ask the participant questions and its face would move in reaction to the participant.
The participants would go through a tutorial, led by the device, to build a basic circuit. They then filled out a ten minute survey about their experiences, and afterwards had up to 15 minutes of exploratory time with the device.
In our results, we found that one setup case was clearly not the "best." We found that the student level of engagement was much higher in the interested cases, and that the students' comfort with the device grew later into the tutorial. We concluded that the big difference between the onboard and the offboard device was that the offboard device acted like a teacher, while the onboard device acted like a peer.
I am third author on a paper accepted to DIS 2014, the Association for Computing Machinery's conference on Designing Interactive Systems.
The onboard version of the device that would lead participants through a tutorial to make a basic circuit.
We had several hypothesis going into the project:
1. The device that expresses interest towards the learner will be perceived as both more likeable and more socially present than the device that does not express interest towards the learner.
2. Participants will perceive the task as less stressful when interacting with an embedded device than when interacting with an external device.
3. Interacting with the interested device will lead to higher confidence about acquired electronics prototyping skills than the non-interested device.
4. Interacting with the embedded device will lead to a higher prototyping confidence than interacting with the external learning agent.
The study setup as viewed by a participant. In this case the device is offboard.
My role throughout the project was to recruit participants, run the experimental trials, and then analyze parts of the data. Our participants were over 70 high school students (age 15-18), most of whom were at Stanford for various summer programs. I recruited the particpants, scheduled them for their time slots, and collected their permission forms from their parent/guardian.
I was one of two researchers running the experimental trials, which took approximately an hour each. We ran over 70 experimental trials in 5 weeks. As it was a "Wizard of Oz" study, I controlled the device and had it respond and speak to the participant.
After the experimental trials were finished, I analyzed our video data for speech length and content patterns for when the student was talking to the device. I also analyzed the exploratory video for patterns on students' focus in exploratory prototyping.
In keeping with our first hypothesis, participants rated the interested device as more likeable than the uninterested device. They also found the interested device to have a higher social presence than the uninterested device.
Device location significantly affected perceived task stressfulness. The embedded device was found to make the task significantly less stressful than the external device. We believe that the embedded device acted more like a peer, helping the student along the way collaboratively, while the external device acted more like a teacher giving instructions. For example, the students interacting with the external agent in total said "I don't know" or some similar variation thereof 8 times, while only one student said "I don't know" when interacting with the embedded agent. The embedded agent seemed to encourage more guessing and thought when students answered the questions.
In the interested cases, the follow up questions that the device asked the student tended to be much more engaging. We believe that students took a while to warm up to the device, but then often ended up enjoying interacting with it after a small amount of time had passed.
The open ended prototyping session was particularly engaging, as we told participants that they didn't have to use all 15 minutes that we gave them, but nobody stopped before 15 minutes were over.
